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This mama is to explain why the Artificial Intelligence 
gToup of Project MAC is developing methods for on-line tracking 
of humm eye movements. It also gives a brief resume of results 
Co date and the next steps. 



1 . 


I b Reasons for d e-ve.1 oc lna. real-time. tracking systems . 

The study o£ eye- motions is important already in perceptual 
psychology -and In. human engine*ting of ratty kinds of systems. Standard 
cethods are based 0 -n. recording, in one way or another, and analysing the 
results later, These methods do not permit experiments making the visual 
environment conditional On the oculomotor behavior,, and are thus Halted 
in the extent of experimental control of the stimulus. 

Standard methods cf eye-tracking all require either immobilization 
of the subject's head, by clamps or bite-bars, or wearing special helmets 
or contact lenses. One of our goals is to allow the subject to operate 
without any special apparatus* Thus, we have to reject special contact 
lenses, which otherwise: would make remote tracking fairly easy, 

how we list reasons for eye-tracking more specifically. 

1.1. I-ve-motions are the only observable int ra-petCteptuA 1 pjrjjccsses . 

The Artificial Intelligence group is concerned with making perceptual 
machinery, and naturally wants to know about human visual mechanisms. 
Unfortunately, too little is known, and this is one instance where it 
appears that we will have to do much of the work ourselves to find out 
what we need. Eye-mo Clone report to us the effect of certain decisions 
in the scene-analysis process; we have no such information in, say, 
audition, {We do. In tactile perception, but the problems here aren't 
so interesting, at least yet). Thus, we ought to be able to find out a 
great deal about the relation between central and peripheral visual field 
cues, by watching eye motions * While this has often been done to soec 
extent with off-line recordings, the results of such analyses are only 
interesting, and net very decisive. With on-line information, we can 
cake the visual stimulus——on a display stupe, for example— depend on 
the sequente of fixations in such ways as to make decisive determinations 
of which peripheral cues has what effects. 
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1 ,2 Clues about Internal thought processes .. 

Allen Newell has suggested a number of ways that eye-motions wight 
be analysed in conjunction with problem-solving "protocols'" so as to help 
trace a subject's execution through his problem-™lying fl™ diagram, 

Although no really conclusive techniques for thiE are established* the 
possibilities seem important, for the reasons discussed in 1.4 below. 

1, 3 Use as Control svstest input . 

It seems likely that One could uge eye-motion for control of complex 
systems, in seme very comfortable and precise ways. The program has to 
infer the subjects intentions from his fixations on a possible very c-cmplex 
control console, and we can expect serious problems in classifying such 
intentions and Teooguizing them. Tracking, in particular, should be 
particularly natural. It should be noted that there is nc reason to expect 
visual cracking control to be comparable to manual control in precisionf 
one must expect CO have to need Sketchpad^like inferences about the proximity 
of the fixation ftOit a tontTol point, and fiom the local behavior of the 
fixation, 

1.6 th'j.otliegis about smooth futefration with, memory . 

Another rtitM for supposing that on-line eyt-tracking will be important 
is based on fhe following Introspective notion. Normally, one is unconcious 
of details about one'a memory of visual scenes, especially when the scene is 
present, Tor then, one can either reMmbct what is in a portion of the 
Visual environment or one can look there. It is possible that these alterna¬ 
tives are integrated into scene analysis so smoothly that instructions to 
move the .ixation point to e paint have practically the same status as requests 
to memory for inforiaatton about what ie {supposed to) be there. If something 
like that holds, them we may be able to make dr&m4Ci-Sally effective and 
theoretically interesting information retrieval system according to the 



following schema: Imagine a display with only some very general categories 
of information on it, Assume that the user is quite familiar with this 
Coarse system, When he fixates -on a region * this means that he is interested 
id it- and that part of the display is replaced by m expansion in greater 
detail* The subject feels that he knew this material already, in his mind's 
eye, and if he is really fhttfliar With the second-level he will find the 
next sub-category in the expanded portion quickly, again without conscious 
effort* The effect, I hope, would be that subjectively the system will feel 
as though an immense amount of detail is available, in familiar forms, in the 
immediate visual environment, although this. is not really true, of course* 

Wo certainly ought to be able to get simple first-order effects ouch as 
television pictures that appear to have full bandwidth, though only a seall 
output dara rate is really Operating* We cannot expect to make the system 
provide new information in forma that make the subject feel that he already 
"knows" it thoroughly, for can't expect to get the effect of full networks 
of relations, but one Ought CO get some such effects and it seems worth find¬ 
ing how far it goes* 

2. Simple immediate aoo1fcaCion5 , 

The following experiments could be immensely important in education, 
if they yield good results. 

2.1 Reading without learfLinft to read . 

Suppose tlxat a pre—literate child, adult or blind parson {with oculomotor 
control) could look at a page, and have the word or phrase in fixation tead to 
him* The eye-tracking program would have to be developed to a point of some 
refinement. of course, to approximate the subject's attention* Then, 

(1) One would expect some learning. 

(li) One would in fact be able to "read". Por the child, thie is 
valuable enough ao that be would want to keep at it. The control experience 
in reading cneself is different from the experience of being read tO- Indeed, 
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U a special skill in Itself, sc that 

fill} the control system for reasing can be developed independent 
of the character-and vord-readlp£ learning. One need net degrade the 
child back to the "primer" vocabulary. 

2 .2 Language learning . 

The problem of learning to read a foreign lifioAge is complicated 
by the need simultaneously to acquire several skills—vocabulary, inflec~ 
tion T grammatical Construction and word-order, pronunciation, and even new 
character sets, Several of these, particularly Vocabulary knowledge, can 
be minimized by having the word, under fixation persists longer than 
threshold. (Or instead of threshold, some other property of eye^motion, 
or any ether subject-output could be u£ud,) Instead of pronunciation, 
one might replace the word by its English equivalent, or if necessary 
Che phrase, or one eight apply same other transformations, Qne could guide 
the reader by programmed Cues, like brightening, moving, enlarging, delet- 
ingj. etc.* sections or phrases ahead or behind, to encourage desired scfttihing 
behavior or other reading skills. 

2.1 Blind , 

¥atvy blind persona have good oculomotor control, or could acquire 
it, given appropriate feedback. Although a blind person car. train hiss elf 
to understand test read at higher than normal rates, this is next-to-useless 
for intricate things like mathematical reading, where one doeitl't normally 
soap the page in a "Tabular 11 manner. The seme Is really true for any dif¬ 
ficult reading, and it it possible that with oculomotor control over a page, 
blind people could absorb the liatarial, read-alound fast but locally, at 
very respectable rates. It would be fun to see if anything could be done 
in the line of making aural picture images, with special sounds. 



